7 research outputs found
AON: Towards Arbitrarily-Oriented Text Recognition
Recognizing text from natural images is a hot research topic in computer
vision due to its various applications. Despite the enduring research of
several decades on optical character recognition (OCR), recognizing texts from
natural images is still a challenging task. This is because scene texts are
often in irregular (e.g. curved, arbitrarily-oriented or seriously distorted)
arrangements, which have not yet been well addressed in the literature.
Existing methods on text recognition mainly work with regular (horizontal and
frontal) texts and cannot be trivially generalized to handle irregular texts.
In this paper, we develop the arbitrary orientation network (AON) to directly
capture the deep features of irregular texts, which are combined into an
attention-based decoder to generate character sequence. The whole network can
be trained end-to-end by using only images and word-level annotations.
Extensive experiments on various benchmarks, including the CUTE80,
SVT-Perspective, IIIT5k, SVT and ICDAR datasets, show that the proposed
AON-based method achieves the-state-of-the-art performance in irregular
datasets, and is comparable to major existing methods in regular datasets.Comment: Accepted by CVPR201
MaskOCR: Text Recognition with Masked Encoder-Decoder Pretraining
Text images contain both visual and linguistic information. However, existing
pre-training techniques for text recognition mainly focus on either visual
representation learning or linguistic knowledge learning. In this paper, we
propose a novel approach MaskOCR to unify vision and language pre-training in
the classical encoder-decoder recognition framework. We adopt the masked image
modeling approach to pre-train the feature encoder using a large set of
unlabeled real text images, which allows us to learn strong visual
representations. In contrast to introducing linguistic knowledge with an
additional language model, we directly pre-train the sequence decoder.
Specifically, we transform text data into synthesized text images to unify the
data modalities of vision and language, and enhance the language modeling
capability of the sequence decoder using a proposed masked image-language
modeling scheme. Significantly, the encoder is frozen during the pre-training
phase of the sequence decoder. Experimental results demonstrate that our
proposed method achieves superior performance on benchmark datasets, including
Chinese and English text images